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Abstract 

We study Sigma-Delta (SA) quantization of oversampled bandlimited functions. We prove 
that digitally integrating blocks of bits and then down-sampling, a process known as decimation, 
can efficiently encode the associated EA bit-stream. It allows a large reduction in the bit-rate 
while still permitting good approximation of the underlying bandlimited function via an appropriate 
reconstruction kernel. Specifically, in the case of stable rth order EA schemes we show that the 
reconstruction error decays exponentially in the bit-rate. For example, this result applies to the 1-bit, 
greedy, first-order EA scheme. 


I. Introduction 

Analog-to-digital (A/D) conversion is the process by which signals (viewed as vectors) are rep¬ 
resented by bit streams to allow for digital storage, transmission, and processing using modern 
computers. Typically, A/D conversion is thought of as being composed of sampling and quantization. 
Sampling consists of collecting inner products of the signal with appropriate vectors. Quantization 
consists of replacing these inner products with elements from a finite set, known as the quantization 
alphabet. Often, quantization is followed by some form of encoding or compression, in order to reduce 
the size or bit-rate of the digital data. A good A/D scheme allows for accurate reconstruction of the 
original object from its quantized (and compressed) samples. Sigma-Delta (SA) quantization was 
proposed in the 1960’s Q as a method for digitizing bandlimited functions. In fact, SA quantization 
schemes remain in use today, in large part due to their robustness to errors caused by circuit 
imperfections, but also due to their ability to trade-off quantizer bit-depth and oversampling (cf. 

El). 

In the context of bandlimited functions, oversampling —coupled with an appropriate SA quanti¬ 
zation scheme— enables one to use coarse (even binary) quantization alphabets, such as A := {±1}, 
and then to reconstruct the function accurately from the resultant bit-stream. In particular, EA 
schemes have been devised Q, ||4l whereby the reconstruction error, measured in the norm, 
decays exponentially fast in the oversampling rate. Specifically, 131 and 0 each devise a family 
of sophisticated EA schemes parametrized by an order r, and choose an appropriate scheme (from 
this family) by optimizing r as a function of the oversampling rate. Working with the alphabet 
A = {±1}, and denoting the oversampling rate by A, the best known reconstruction error guarantees 
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(see 11) behave like 2 with c « 0.1. In this context, since the size of the alphabet is fixed, the 
bit-rate resulting from SA quantization is proportional to the oversampling rate. Consequently, the 
reconstruction error of Q and H decays exponentially fast with the bit-rate, albeit with a sub-optimal 
coefficient in the exponent^ 

In this note, we prove that using any stable rth order SA schemes, with an arbitrary integer r > 0 
(including the 1st order, greedy, SA scheme defined below) followed by a simple encoding step, 
we can always reconstruct a bandlimited function from its encoded bit-stream with a reconstruction 
error that decays exponentially fast in the bit-rate. Moreover, we obtain a near-optimal coefficient in 
the exponent. 


A. Preliminaries 

We define fhe Fourier fransform, /, of / S L^(M) via 


OO 



— OO 


The inverse Fourier transform is then given by 

OO 

/(*) = ^ j 

— OO 


In this note we are interested in bandlimited functions / S L^(R) with |/(t)| < 1 and with Fourier 
transform vanishing outside the interval [—7r,7r]. We denote the set of such functions by B.,^. The 
classical sampling theorem yields a method of reconstructing an arbitrary f £ perfectly from its 
so-called Nyquist rate samples f{n),n G Z. In particular. 




sin(7r(f — n)) 
7r(f — n) 


( 1 ) 


Nevertheless, sampling at this Nyquist rate is rarely done in practice because the reconstruction kernel 
decays too slowly. This implies that if one were to reconstruct with “noisy” samples 
/(n) + £n (instead of with /(re) in ([T])) large, possibly unbounded, reconstruction errors could result, 
even if e„ were bounded. This makes ffl unsuitable for reconstruction from quantized samples. 
Instead, one may revert to oversampling, i.e., collect the samples f{n/X) for some A > 1 and then 
reconstruct via the formula 

fit) = j^f{n/X)git-n/X), (2) 

nEZ 


where p is a function with g G C°°, giu) = -^= for |u;| < vr and g{uj) = 0 for |a;| > Avr. With these 
sampling and reconstruction schemes, it can be seen (cf. Q) that the reconstruction error induced 
by small errors in the sample values is small. In the worst case, it is proportional to the error in the 
samples. On the other hand, in the quantization setting one has control over how the samples /(re/A) 
are replaced by elements from A, so one can do significantly better. 


'For example, given a bit-rate of A bits per Nyquist interval, one can obtain exponential decay in A (with a much better, 
essentially optimal (see, e.g., 0), coefficient in the exponent) hy sampling at slightly higher than the Nyquist rate and 
replacing the samples hy their binary approximations. In particular, the error is 0(2“"''). On the other hand, this method 
is not robust to errors in assigning the bits (cf. t2ll. 
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B. SA quantization and prior work 

One-bit, first order, greedy SA quantization produces bits G {—1,1} vi£§| the following 
recursion, with initial condition mQ) I'^^ol < 1- 


(In^ = sign {un-i + f{n/X )), 

(3) 

Un = Un-l + f{n/X) - 

(4) 


One can see, by induction, that |rt„| < 1 for all n. Moreover, using this scheme for quantization and 
the function p in Q for reconstruction, we have (see m) 

\f{t) - - n/\)\ < Cg/\. 

To generalize the above SA scheme, let r be a positive integer and denote by p : —)• M the 

“quantization rule”. One can then define an rth order SA scheme via fhe recursion: 


= sign (//(/(n/A), Un-l,'U„_ 2 , • • ■ ,Un-r)) , 

(5) 

(A^'U)„ = /(n/A) 

(6) 

where the operation of the difference operator A on a sequence h is defined by {Ah)n 
® is equivalent to 

•— hfi 

Un = /(n/A) - qW - ^ {-lyun-j. 

7 = 1 

(7) 

An important issue in the design and analysis of higher order schemes is ensuring that the sequence 
Un is uniformly bounded via a proper choice of p. Thus, we say that an rth order SA scheme is stable 
if n oo < C^A whenever /(n/A) < 1 for some constant Csa that may depend on r. Daubechies 
and DeVore 13 proposed the first family of stable SA quantization algorithms and used them to 


obfain error bounds of the form 

\f{t)-'^Qn'’9{t-n/X)\ <C{r)/\\ 

By choosing the optimal r(A), they also derived the improved estimate 

l/W-E 

nEZ 

Guntiirk l|3l proposed a different family of SA schemes and used them to obtain the bound 

l/W -E -"/A)l<2-G 

nEZ 

with c 0.07, again by choosing the order r as a function of A. Deift et al. improved this result 
by obtaining the coefficient c « 0.102 in the exponent. 

For the case of constant input to the SA quantization, there has been some work (cf. lISl-lTTl) 

^Here and throughout, we use the superscript A to indicate the oversampling rate at which a discrete sequence is obtained. 
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seeking upper bounds on the number of possible SA bit-sequences of length N. For example Q 
showed that asymptotically, for first-order SA schemes, the number of such sequences is 0{N‘^). 
These sequences can be represented by binary labels of length 0(log(A)) while still enabling a 
reconstruction error of 1/N. However, no analogous bound is known for bandlimited functions. 

In practice, when working with oversampled A/D conversion of bandlimited functions, it is common 
to incorporate a so-called decimation step (see, e.g., ||8l). This process reduces the bit-rate by 
mapping blocks of quantized samples (obtained at a high oversampling rate) to elements from a 
codebook (another finite set). An analysis of such techniques was given by Candy ||8l, under the 
simplifying (albeit generally false) assumption that SA quantization introduces random “noise” that 
is uncorrolated with the input. The conclusion of lEl, based on the randomness assumption and 
numerical experiments, is that decimation can produce dramatic decreases in the bit-rate without 
compromising the quality of approximation. In this note, we provide a rigorous mathematical analysis 
of decimation, with the same conclusion. 


II. Main result 


We prove that by digitally integrating blocks of bits produced by one bit, rth order, stable SA 
schemes —a process known in the engineering community as decimation [Si — we can reduce the 
number of bits per Nyquist interval from A to approximately r log A. We prove that this still allows 
for an approximation error that decays like l/W, albeit via a different reconstruction kernel g than 
that of In other words, we show exponential decay of the approximation error as a function of 
the bit-rate, with a near-optimal exponent. 

To make the discussion more concrete, let us start with some definitions. For a sequence h, and 
positive integers r, p > 1, define fhe rth order partial sums 

(s;/l)n := jz {Sl-'h)n-m 

^ m=—p 


1 

2p + l 


n+p 

E <«?■''>) 

m=n—p 


where {Sph)n ■= hn- For a bit-sequence generated from an rth order EA quantization of a 
bandlimited function, and for an integer p < we are interested in the integrated bit sequence 
as well as its decimated (subsampled) version 


Hn 


We prove the following theorem. 



(2p+l)n 


Theorem 1. Suppose that f is in , p £ N n (1, and define := 2 p+i - Then the following 
are true of 1-bit stable rth order SA quantization. 

(i) There exists a function g such that 

-n/X') - f{t) 


<CsaC'^(^)’'=:P. 
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(ii) To encode \ one needs at most A' log 2 ^(2p + l)'^ + 1^ bits per Nyquist interval where 


^ log 2 ((2p + 1)^ + l) < A' log 2 (2^—^ ) —: TZ. 


Consequently 


V{n) = 2CsaC^2-^/^'. 


( 8 ) 


Here C > 1 is a constant independent of A, A' and r. Csa a constant that depends on the scheme 
(i.e., possibly on r). 

Remark 1.1. As A grows, we may select a progressively larger p, so that in the limit A' approaches 
1. Hence the claim about near-optimality. 

Remark 1.2. Examining the proof of the theorem (helow), one should he able to extend the proof 
without too much difficulty to the case of multi-bit quantization. For ease of exposition, we refrain 
from doing this in this note. 


III. Proof OF Theorem [I] 

Proof: We will begin by proving (i). Our goal is to bound the error 


e := 


- n/X') - f{t) 


(9) 


To that end, let us first define the sequence := f{n/X). Using the triangle inequality, we 

have e < ei -f 62 where 


61 := 


V E - (■s;/‘"’)(2p+l)n)9(« - "/V) 


1 

A 


nG'Z 


(2p+l)n 


■g{t-n/X') 


and 


62 := 


^EWU')(2,+ .)n'9(‘ 


n/X') - f{t) 


The remainder of the proof will consist of bounding 6 i and showing that there exists a function g 
for which 62 = 0. Along the way we will specify g. 

To bound 61 , we first define, for an integer p, the difference operators Ap, and Ap by their 


actions (^Apxj := Xn+p — Xn-p-i and (^Apxj := Xn — Xn+ 2 p+i, respectively. One easily checks 
that 5pA = 2 ^^Ap and similarly S'pA^ = ( 2 p+i)^ where A^ := (Ap)^. For convenience, we 
introduce the notation := g{t — n/X') and observe that {gl^^)^ = {9t^^){2p+i)n- Using the 

SA state equations (l5]l, ©, and then reindexing we can write 
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The last inequality is due to (the proof of) Proposition 3.1 in O; the notation stands here for 
the rth derivative of the function g. We shall now turn to controlling 62 , and return to the right hand 
side of (fT^ shortly. 

To hound 62 , let us first extend the use of our notation for partial sums so that for integers r > 1, 
{S^f){t) := 2 ^^ ~ m/X) where {S^f){t) = f{t). Thus, taking Fourier transforms 


Let h{ui) G C°° satisfy 


sin(g^tu) 

( 2 p + l)sin(^a;) 


/(^) 


sin(^cu) 
T sm(^a;) 


/(w). 


(13) 


h{uj) : 


= 


1 , 


tu = 0 


I, ,1 < TT 

\. • / 1 \ * Uj TT 

|a;| > A'vr. 


2 \' 

0 , 


Since / is compactly supported, using Fourier series we have 

(^(a;) = 




where g G (7°°, p(w) = for |a;l < vr, g{uj) = 0 for |a;l > A'vr. Flere, 


ttA' 


Cn. — 




y/^X 


—ttA' 


Thus, we deduce that 


ttA ttA 






{S^f){u;)h{ujYe^^^duj 


ttA 


J n&L 

— TTA 


nez 


ttA 


—ttA 


(14) 


(15) 


(16) 


(17) 


(18) 


Let /ir(f) he the inverse Fourier transform of h^iio) and denote hy g{t) := {g*hr){t) the convolution 
of g and hr- We now have that f{t) = ^ Y2n&z ^2 = 0. To conclude 

the proof of (i), we note that * ^rlUi < IIli ll^rllii < C'^A'*' where the last 

inequality is a direct consequence of Lemma |2] helow and the fact that ||li can he treated as a 
constant. Noting that ||u||oo < C'sa completes the proof. 

To prove (ii), note that the sum of 2p+1 elements each taking on values in {±1}, is an odd integer 
in [—(2p + 1), 2p + 1]. There are 2p + 2 such integers, so each element of the sequence can 

he encoded using log2(2p + 2) hits. Similarly, the sum of 2p + 1 odd integers in [—(2p + 1), 2p + 1], 
is an odd integer in [—(2p + 1)^, (2p + 1)^]. There are {2p + 1)^ + 1 such integers. Proceeding in 
this fashion, we see that each q!^ ^ can he encoded using log 2 ((2p + 1)'" + l) hits. Moreover, note 
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that due to decimation, for every A original SA bits of there are A' = 2 ^^ elements of . 
The rate-distortion relationship then follows by combining (i) and (ii). □ 


Appendix 


Lemma 2. Let 4>o{^) be in C°° and bounded, with 4 >o{uj) = 1 when |a;| < 1 and (/>o(w) = 0 when 
|w| > c for some fixed c G (l,oo). Define := <^ 0 ( 7 )- het X' > c, let ^o(w) = .y 2 in (\ ' 
define 


h{uj) := ho{u)fi{u) = ho{u)fio{u/' k). 


(19) 


Then ||/i||li = \ h{t)\dt < CX' where C depends on cpQ. Consequently, for any r > 1, denoting 

by hr{t) the inverse Fourier transform of h^{uj), we have ||/ir||Li < C'^A''’. 


Proof: Note that 


\h{t)\dt = 


7-00 + 1 7-A' 


f*A'7r 




dt 


+ 


1 /■°° 1 

7-00 + 


f*A'7r 


Ph{u})C^^duj 


dt. 


( 20 ) 


^/^ 7-00 +1 7 

We will proceed by bounding each of the summands on the right hand side separately. The first term 
is controlled by 


1 


v^7- 

< 


.00 ^ 

7-00 +1 


pA '71 


'-A'- 


h{oj)e^‘^^du} 


dt 


1 /■°° 1 o„ , 


7-00 + 


£2 ^ C'^qA 


where = sup |(^o(<^)| and the first inequality is due to the bound |/io(w)| < 7r/2 when |w| < A'vr. 

U) 

To control the second term, we observe that 

t‘^h{uj)C^^duj 


1 


/: 


1 


^/^ 7-00 + 1 

poo 


dt 


< 


< 


hi 

— f 

7-00 +1 7-A' 


7-00 + 1 7-A'7r 

poo ^ pX'tt 


h{uj){C^yduj 


dt 


k'{u>){C‘^^)duj - 


1 r°° 1 / \ 


dt 

dt. 


7-00 +1 ^ 7-a'- 

Above, the first inequality is due to integration by parts. In particular, 

t " h{u){C^ydu = h{uj){C^^)'\^ll,^ - h'du 

J—X'tv J — X'tv 

= 0 - h'{u>)C^Yx'^ + t" h"{oj)iC^^)doj. 

J—X'tt 


( 21 ) 



































Thus 


1 r 


1 


r*A'7r 


J-OO + 1 7-A't 

1 


< 


-T 


7-00 + 1 

< v^(A'7rC^„ + q,), 


dijj dt 

(2A'7rC^„ + 2C^,)dt 


( 22 ) 


( 23 ) 


A' sm( 5 ^) cos(^)-A sin(^) cos(^) 


achieves 


where the constants satisfy C^,, > |ii,"(a;)| for all ui G [—A'tTjA'tt] and > \h'{io)\ for all co G 
[—A'tt, A'tt]. 

To compute Cf^,, we observe that the function h'^lio) :=- ^ -y 

its maximum absolute magnitude on [—A'tt, A'tt] at ±A'7r. Denoting this maximum by Ci, we have 
Cl = < i, since A' > 1. Similarly, one can verify that ^o(w) achieves its maximum amplitude 

on [—A'tt, A'tt] at zhA'vr. A simple evaluation then reveals that the maximum, denoted by C 2 , is 

ginf-2^ ) ( —A'^ ) 

--. Using that sin(x) < x, we observe that C 2 < vr/8. Next, observe that = 

00 (w/ A'tt) thus 

|/r'(a;)| = |(/io(o.)0(a;))'| 

< |/io(a;)0o(<A;/vr)| + |/io(a;)0o(a;/7r)/7r| 


1 ^ TT U, 
< 2^00 + 2 • - 


TT 


( 24 ) 


where = sup |0Q(ti;)|. Similarly, 

CJ 

|/r"(a;)| = |(/roM0M)"| 

< |/io(a;)0o(w/7r)| + |2/io(a;)0o(w/7r)/7r| 

+ l^o(w)0o(a;/7r)/(7r)^| 

< — r 4- ^*^0 4_ — 

- 8 • Wo T ^ + 2 ■ (7r)2’ 

where = sup|0q(w)|. Substituting the above bounds on \h'{Lo)\ and \h"{uj)\ into (|2^ and 

LJ 

then combining the result with (|2T1) and (l20l) yields the desired result on ||/i||Li. The statement on 
||/ir||Li follows by observing that hr is the convolution of h with itself r times. As h is in Li, 

Whrh. < \\h\\l^. □ 
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